Moving object detection, tracking, and displaying systems

ABSTRACT

Moving object detecting, tracking, and displaying systems are provided. Systems illustratively include a graphical user interface and a processing unit. The processing unit is a functional part of the system that executes computer readable instructions to generate the graphical user interface. The graphical user interface may include an alert and tracking window that has a first dimension that corresponds to a temporal domain and a second dimension that corresponds to a spatial domain. In some embodiments, alert and tracking windows include target tracking markers. Target tracking markers optionally provide information about moving objects such as, but not limited to, information about past locations of moving objects and information about sizes of moving objects. Certain embodiments may also include other features such as zoom windows, playback controls, and graphical imagery added to a display to highlight moving objects.

REFERENCE TO RELATED CASE

The present application is based on and claims priority of U.S.provisional patent application Ser. No. 61/265,156, filed Nov. 30, 2009,the content of which is hereby incorporated by reference in itsentirety.

BACKGROUND

Many situations exist in which it may be useful or desirable to be ableto detect and track moving objects. Two examples of such situationsinclude security and surveillance applications such as for border orport security.

Moving object detection and tracking has historically been a relativelylabor intensive process. Human operators would commonly have todiligently watch an area. The required attention could limit the amountoperators can effectively observe.

Some semi-automated systems have been developed to aid operators insecurity and surveillance applications. One system includes highlightinga moving object by overlaying a box around the moving object in theoriginal image. Some systems have also included showing a “tail” in theoriginal image that provides an indication of a past location of theobject.

SUMMARY

An aspect of the disclosure relates to systems for detecting, tracking,and displaying moving objects. Systems illustratively include agraphical user interface and a processing unit. The processing unit is afunctional part of the system that executes computer readableinstructions to generate the graphical user interface. The graphicaluser interface may include an alert and tracking window that has a firstdimension that corresponds to a temporal domain and a second dimensionthat corresponds to a spatial domain. In some embodiments, alert andtracking windows include target tracking markers. Target trackingmarkers optionally provide information about moving objects such as, butnot limited to, information about past locations of moving objects andinformation about sizes of moving objects. Certain embodiments may alsoinclude other features such as zoom windows, playback controls, andgraphical imagery added to a display to highlight moving objects.

These and various other features and advantages that characterize theclaimed embodiments will become apparent upon reading the followingdetailed description and upon reviewing the associated drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a screenshot of a moving object detection and trackinggraphical user interface.

FIGS. 2-1, 2-2, 2-3, and 2-4 are screenshots that illustrate an alertand tracking window being populated with data over time.

FIG. 3 is a screenshot of a graphical user interface having zoomwindows.

FIG. 4 is a block diagram of an operating environment that may be usedto implement moving object detection, tracking, and displaying systems.

FIG. 5 is a process flow diagram of one method of detecting movingobjects.

DETAILED DESCRIPTION

Embodiments of the present disclosure include systems and methods fordetecting, tracking, and displaying moving objects. One notable featureof certain embodiments is an alert and tracking window. Alert andtracking windows are illustratively placed below or near a video displayof the area being monitored. In at least certain situations, alert andtracking windows aid operators in quickly and effectively detecting andtracking moving objects. Alert and tracking windows may include trackingmarkers that provide an indication of past locations, direction oftravel, speed, and relative size of moving objects. These features maypermit operators to observe a larger area than they otherwise could.Also, as is discussed below in greater detail, some embodiments may alsoprovide additional benefits and advantages such as, but not limited to,reducing costs, enhancing operator sensitivity in identification ofmoving objects, and helping operators distinguish between moving objectsof interest (e.g. a boat) and clutter or noise (e.g. a tree withbranches or leaves moving in the wind).

FIG. 1 is a screenshot of a moving object detection and trackinggraphical user interface (“GUI”) 100. GUI 100 is illustratively shown ona display device and monitored by one or more operators. It includes avideo window 102 and an alert and tracking window 152. Video window 102illustratively shows real-time or approximately real-time video of anarea being monitored. Window 102 includes a height or vertical dimension104 and a width or horizontal dimension 106. Embodiments of the presentdisclosure are not limited to any particular dimensions 104 and 106, andillustratively include any dimensions 104 and 106. Some examples ofpossible dimensions for illustration purposes only and not by limitationinclude “standard” video dimensions of 4:3 or 16:9 and panoramicdimensions such as, but not limited to, 108:7. Video window 102similarly also optionally includes any number of pixels in itshorizontal and vertical directions.

Video window 102 illustratively includes target detection highlights.Target detection highlights are graphical features overlaid on top ofthe video that indicate that an object shown in the video is moving. Inthe specific example shown in FIG. 1, the target detection highlightsinclude colored squares that are placed around boats 111, 112, 113, and114. Target detection highlights are not however limited to anyparticular configuration and optionally include any implementation suchas circles, dots, change of shading or coloring, added graphical figuresoverlaid around or near the moving objects (e.g. boxes), etc. In oneembodiment, target detection highlights only depict the current locationof a moving object. However, in another embodiment, target detectionhighlights may depict both the current and past locations within thevideo window. For example, past locations of a moving object areillustratively shown by a “tail.” Each point in the tail represents ahorizontal and a vertical coordinate of the moving object in the past.The intensities of the points along the tail are optionally decreasedsuch that the more recent locations are brighter than older locations.In one specific example, intensities are decreased by ten percent foreach refresh or update. This results in the ends of the “tails” (i.e.the oldest parts) eventually going away so that the video window doesnot become “cluttered” with “tails.” Also, it should be noted thatalthough several examples of target detection highlights have beendescribed, that embodiments of the present disclosure are not limited toany particular manner of showing current and/or past locations withinvideo windows and embodiments illustratively include any manner ormethod.

Alert and tracking window 152, as is shown in FIG. 1, is illustrativelypositioned beneath video window 102. Alert and tracking window 152 mayhowever be optionally located at any other position relative to videowindow 102 such as above it, to its left or right sides, or at adiagonal. Alert and Tracking window 152 has a height or verticaldimension 154 and a width or horizontal dimension 156. Verticaldimension 154 illustratively corresponds to a temporal dimension, andhorizontal dimension 156 illustratively corresponds to a spatial orgeographical dimension.

Alert and tracking window 152 includes a number of rasters. Each rasterillustratively runs along the entire width 156 of the window and has aheight that is a fraction of the height 154 of the window. Each rasteralso illustratively includes a number of columns that are a fraction ofthe width of the raster. In one embodiment, the heights of the rastersand the widths of the columns are illustratively one pixel, butembodiments are not limited to any particular number of pixels ordimensions.

In an embodiment, each raster in alert and tracking window 152corresponds to one video frame of the video being shown in video window102. For instance, the raster at the top 161 of window 152illustratively corresponds to the current video frame being displayed invideo window 102, and the raster at the bottom 162 of window 152corresponds to a video frame that has been previously shown in window102. Embodiments are not however limited to a one-to-one correspondencebetween video frames and rasters in alert and tracking window 152.Rasters in alert and tracking window 152 may be generated at a fractionof the video frame rate. For instance, rows may be generated once forevery ten video frames, once every one hundred video frame, or any otherfrequency. Additionally, the height 154 of window 152 is not limited toany particular dimension. Accordingly, window 152 illustratively hasrasters that correspond to any number of previous video frames that maybe desirable. For instance, for illustration purposes only and not bylimitation, window 152 could be set-up to show information thatcorresponds to 10 minutes, 30 minutes, 1 hour, 2 hours, or any otherlength of time.

In one embodiment, the intensity of the highlighted pixels or areas in araster is dependent upon the number of pixels or amount of area in thecorresponding video window for which motion is detected. The intensityillustratively increases as the amount of detected motion increases. Forexample, a column in a raster corresponding to five pixels with detectedmotion is displayed brighter than a column in a raster onlycorresponding to three pixels with detected motion. Accordingly, in someembodiments, alert and tracking windows provide users with indicationsof sizes or at least relative sizes of moving objects. An indication ofhorizontal size is given by the number of highlighted columns in araster, and an indication of vertical size is given by the intensity ofthe highlights within the columns.

The width 156 of alert and tracking window spatially corresponds towidth 106 of video window 102. In the particular example shown in thefigure, there is a one-to-one correspondence, but embodiments includeany ratio of correspondence. In the example shown in FIG. 1, each columnin a raster corresponds to the column of pixels vertically above it invideo window 102. As an object shown in video window 102 is detected asmoving, the area or pixels in the corresponding location in the alertand tracking window are highlighted. For instance, in FIG. 1, boat 111is detected as moving. Accordingly, the corresponding area or pixels 171along the horizontal direction of alert and tracking window 152 arehighlighted. The figure similarly shows that areas or pixelscorresponding to boats 112, 113, and 114 have similarly beenhighlighted. Other areas of alert and tracking window 152 have not beenhighlighted and this indicates that there has not been any motiondetected in the corresponding area of video window 102.

In at least certain embodiments, alert and tracking windows includetarget tracking markers. Target tracking markers illustratively includehighlighted pixels or areas of multiple rasters within an alert andtracking window. The target tracking markers correspond to detectedmoving objects shown in video window 102. In FIG. 1, alert and trackingwindow 152 includes a target tracking marker 170 that corresponds toboat 111. Target tracking marker 170 includes an ending or most recentportion 171 and a beginning or oldest portion 172. As was previouslymentioned, the most recent portion 171 corresponds to the currentlocation of boat 111 shown in video window 102. The oldest portion 172,as well as all of the other portions of target tracking marker 170,correspond to past locations of boat 111. Accordingly, target trackingmarker 170 shows that boat 111 has been moving from the right hand sideof video window 102 towards the left hand side of video window 102. Thealert and tracking window shown in FIG. 1 also includes additionaltarget tracking markers such as marker 182 that corresponds to boat 112,marker 183 that corresponds to boat 113, and marker 184 that correspondsto boat 114.

Before continuing to discuss additional features of embodiments of thepresent disclosure, it is worthwhile to highlight a few other benefitsand possible advantages of features that have been discussed. Onebenefit of such a system is that it allows for operators to easilyidentify moving objects in video window 102. For instance, an operatorwalking up to video window 102 may not be able to easily discover thatboat 112 is moving or even that boat 112 exists. However, alert andtracking window 152 provides target tracking marker 182 that is easilyidentifiable by an operator and clearly shows that there has been ahistory of motion. An operator can identify marker 182 and then look tothe corresponding areas of video window 102 to find boat 112. The targettracking marker in this sense helps to narrow or reduce the amount ofvideo window 102 that an operator needs to study to find boat 112.

Another benefit is that alert and tracking windows provide direction andhistorical location information. For instance, in FIG. 1, an operatormay identify that boat 111 is moving. The operator may however have justdiscovered this fact and would not know where boat 111 has previouslybeen. Target tracking marker 170 shows that boat 111 has been movingfrom right to left. The past location information may also be useful inpredicting a moving objects current direction. For example, althoughboat 111 could change its direction, alert and tracking window 152 showsthat boat 111 has been heading in more or less the same direction for aperiod of time, and thus there is a probability that boat 111 iscurrently still heading in that same direction.

Related to the location and direction information discussed above,another potential benefit is that alert and tracking windows may alsoprovide relative speed information. For instance, if a target trackingmarker is sloped such that it has a large angle separating it from beingvertical (i.e. as a target tracking marker becomes more horizontallyoriented), this indicates that the object has moved across a largerdistance in a shorter amount of time and is thus moving relativelyquickly. Additionally, based on the speed information and the predicteddirectional information discussed previously, in one embodiment, systemsare able to predict future locations of moving objects. These futurelocations can be shown for instance by extending a target trackingmarker past a moving object's current location. Embodiments are nothowever limited to any particular method of communicating possiblefuture locations.

One other benefit worth noting at this point is that certain embodimentsmay help operators distinguish between real moving objects of interestand objects that have been detected as having motion but that are not ofinterest (i.e. clutter or noise). For instance, an area being monitoredmay have an object such as a tree or a water wave that mayintermittently move. Detection systems may detect this movement andhighlight it in an alert and tracking window. The motion however in notbeing constant will only show up as an isolated highlight or mark in thealert and tracking window. These types of highlights in the window aredistinguishable from the more or less solid or constant lines or trailsof target tracking markers. Accordingly, even in systems that havemotion clutter or noise, operators are able to identify objects ofinterest.

Another type of noise that may be present in a system is noise caused bycamera or sensor instability. A camera could for instance be bumped orotherwise moved. As is explained later in greater detail, in oneembodiment, movement is detected by comparing pixels. In such a system,any camera/sensor instability may show up as highlightedpixels/movement. One example of such highlighted pixels caused by camerainstability is the white horizontal line 130 in FIG. 1. As can be seenin FIG. 1, a large portion of the raster at line 130 is highlighted. Inother systems, camera/sensor instability could potentially lead to afalse detection of a moving object of interest. However, in at leastsome systems having alert and tracking windows, such as window 152 inFIG. 1, operators are more readily able to distinguish betweencamera/sensor instability and a real object of interest. This is yetanother benefit provided by certain embodiments.

Returning to describing features of FIG. 1, graphical user interface 100optionally includes playback controls 190. As will be discussed later ingreater detail, embodiments of GUI 100 are illustratively implementedwith systems that have video data storage capability. In such systems,playback controls 190 allow a user to control the video. For instance,an operator may notice that the alert and tracking window includes atarget tracking marker but that the object is no longer present in themost recent video frame. The user may wish to “rewind” the video to goback and watch the video when the target tracking marker was beinggenerated. Playback controls 190 provide for such functionality. In theexample shown in the figure, playback controls 190 include a playforward in real time button 191, a pause button 192, a play in reversein real time button 193, a play forward in quicker than real time button194, a play in reverse in quicker than real time button 195, a skip tothe end of the recorded video button 196, a skip to the beginning of therecorded video button 197, a loop or repeat button to continually replaya user defined portion of the video button 198, and a time dimensionscroll bar 199 that allows a user to select a portion of the time toplayback. Embodiments are not however limited to the particular controlsshown in FIG. 1 and include any variety of user actuable controls orother methods of controlling recorded video playback. Other controlsincluded in some embodiments, for illustration purposes only and not bylimitation, are provided to zoom, add notes or annotations, extract andsave video clips, scroll spatially for wide or panoramic images, archivedata and supplemental information, and to change contrast.

FIGS. 2-1, 2-2, 2-3, and 2-4 show additional embodiments of a graphicaluser interface 200 for moving object detection, tracking, anddisplaying. Similar to GUI 100 in FIG. 1, GUI 200 illustrativelyincludes a video window 202 and an alert and tracking window 252. Onenotable difference between GUIs 100 and 200 is the aspect ratio. GUI 100has more or less of a “standard” aspect ratio or field of view (i.e.common field of view for video capturing equipment). GUI 200 has apanoramic or wide area field of view. As can be seen in the figures, thewidth 206 of GUI 200 is many multiples of the height 204 of GUI 200.Embodiments of GUIs according to the present disclosure include anyaspect ratios or field of views. However, it is worth noting that in atleast some circumstances, some embodiments may be even more beneficialin the context of a wide area field of view GUI. For instance, withoutalert and tracking windows, it may be difficult for operators to monitorthe large area being shown in video window 202. However, with alert andtracking windows, operators can easily identify moving objects ofinterest using the target tracking markers.

FIGS. 2-1, 2-2, 2-3, and 2-4 illustrate how an alert and tracking windowis populated with or filled with data over time. FIG. 2-1 shows alertand tracking window 252 relatively soon after being started up. There isonly a little bit of data being shown at the top 261 of the window. Thisis represented by the narrow line of highlighted pixels or areas towardstop 261.

FIGS. 2-2, 2-3, and 2-4 then show alert and tracking window 252progressively displaying more information as time passes. As can be seenin the figures, in going from FIG. 2-1 to FIG. 2-4, the area between top261 and bottom 262 has progressively more highlights, including targettracking markers 270 of which four are labeled in FIG. 2-4. In theparticular examples shown in the figures, FIG. 2-1 has 1 raster of data,FIG. 2-2 has 10 rasters of data, FIG. 2-3 has 50 rasters of data, andFIG. 2-4 has 100 rasters of data.

FIG. 3 is a screenshot of another embodiment of a graphical userinterface 300 for moving object detection, tracking, and displaying.Like GUIs 100 and 200, GUI 300 includes a video window 302, and an alertand tracking window 352. GUI 300 also includes zoom window features thatare optionally included in embodiments. In the example shown in thefigure, GUI 300 includes zoom windows 310, 320, and 330. Zoom windowsillustratively display a portion (i.e. a narrower field of view) of thevideo being shown in video window 302. In an embodiment, users are ableto select a moving object being shown in video window 302, and upon theselection, a corresponding zoom window with the moving object isgenerated and displayed on the GUI. For instance, in FIG. 3, zoom window310 corresponds to moving object 315, zoom window 320 corresponds tomoving object 325, and zoom window 330 corresponds to moving object 335.Zoom windows are alternatively displayed in either full resolution orwith digital zoom. In an embodiment, a user is able to choose betweenfull resolution or digital zoom, and able to alternate between the two(e.g. start at full resolution and then use digital zoom to zoom in onan object of interest).

In another embodiment, instead of zoom windows being generated basedupon a user selecting a moving object shown in video window 302, zoomwindows are alternatively automatically generated once an object isdetected. Zoom windows 310, 320, and 330 are for instance in anembodiment automatically generated by a software algorithm without auser manually clicking on any objects within video window 302.Furthermore, an automatic tracking function is optionally utilized tokeep moving objects within zoom windows centered within their associatedzoom windows.

As is shown in FIG. 3, multiple zoom windows are capable of beinggenerated and displayed concurrently. FIG. 3 shows the three zoomwindows 310, 320, and 330 being displayed simultaneously. Embodimentsare not however limited to any particular number of zoom windows and mayinclude none, one, or any number greater than one of zoom windows. Onepotential benefit of such a system is that it allows for multipleoperators to simultaneously and separately conduct detailed examinationand manipulation of any portion of the total video image (e.g. viewwindow 302) without interference among those separate operations. Thismay be useful for example to support situational awareness over largeareas such as for border or port security.

GUI 300 also optionally includes features that facilitate identificationof which part of the video window 302 that each zoom window correspondsto. In the embodiment shown in FIG. 3, each zoom window has codedborders surrounding the zoom window and a corresponding coded zoomwindow location indicator above the video window 302. The coded zoomwindow location indicators are illustratively positioned verticallyabove the area in the video window 302 that includes the area shown inthe zoom window. For example, zoom window 310 has borders 311 and alocation indicator 312. Borders 311 and indictor 312 illustratively havethe same color, shading, graphic, number, or any other type of codingsuch that operators are able to recognize which zoom window correspondsto which location indicator. Similarly, FIG. 3 also shows that zoomwindow 320 has borders 321 and location indicator 322, and zoom window330 has borders 331 and location indicator 332. Each zoom window borderand its corresponding location indicator are illustratively coded thesame as each other but differently than any other border or locationindicator being displayed concurrently. It is worth noting that althoughsome embodiments include the features described above, other methods ofcorrelating zoom windows to specific portions of video windows exist,and embodiments are not limited to any particular method or set offeatures.

Zoom window borders may also include other features to aid operators inperforming security and/or surveillance operations. In one embodiment,borders such as borders 311, 321, and 331 include horizontal and/orvertical scrollbars. The scrollbars allow operators to change the partof the larger image (i.e. the image shown in window 302) that is beingdisplayed in the zoom window. An operator could for example change thezoom window to be showing an area to the left, right, above, and/orbelow the current area being shown. In another embodiment, borders alsohave interface elements that allow operators to quickly change the levelof magnification of the zoom window (i.e. an operator can zoom in orzoom out on an area). It should be noted however that embodiments arenot however limited to the specific examples described above andillustratively include any mechanisms for scrolling and zooming. Forinstance, for illustration purposes only and not by limitation,scrolling and zooming capabilities could be implemented utilizingkeyboard strokes or mouse buttons.

FIG. 4 is one example of an operating environment 400 in which someembodiments may be incorporated in. Embodiments are not however limitedto any particular operating environment and are illustrativelyincorporated in any number of operating environments. Operatingenvironment 400 includes display device 402, moving object sensor 404,input device 406, and a computing device 408. Graphical user interfacessuch as those shown in FIGS. 1, 2-1, 2-2, 2-3, 2-4, and 3 areillustratively displayed on display device 402. In an embodiment,multiple display devices are used to display GUIs. For instance, in anembodiment in which the video is wide area view video, multiple displaydevices may be used to show the entire image.

Users illustratively interact with the GUI on display device 402 usinginput device 406. Input device 406 may be one or more devices. Someexamples of potential input devices include keyboards, mice, scrollballs, and touch screens. Embodiments are not however limited to anyparticular type of input devices and illustratively include any type ofinput device.

Operating environment 400 collects information to detect moving objectsutilizing moving object sensor 404. In the examples described above, theinformation that has been collected has been shown as being video from acamera. Embodiments of the present disclosure are not however limited toonly sensors that are cameras. Embodiments include any type of sensorthat is able to collect information that may be utilized in detectingmoving objects. For example, embodiments of sensors 104 include anyimaging sensor that is responsive to different types of radiation. Somepossible types of sensors, for illustration purposes only and not bylimitation, include electro-optic, infrared, radar, magnetic, seismic,and acoustic.

Computing device 408 is illustratively communicatively coupled to movingobject sensor 404, display device 402, and input device 406 through itsinterface 412. Computing device 408 includes memory unit 410 andprocessing unit 414. Memory unit 410 illustratively stores informationor data collected by sensor 404. The stored information can be used forany of a great variety of applications such as, but not limited to,reviewing previously recorded/collected data. In one embodiment,playback controls such as playback controls 190 in FIG. 1 are used toselect which information is chosen for playback and how it will beplayed back (e.g. full-speed, slow motion, etc.). Memory unit 410 alsoillustratively includes computer executable instructions that generatethe GUIs and provide the other features in the above describedembodiments. Memory unit 410 may include any type of memory such as, butnot limited to, hard disk drives, solid state drives, flash memory, RAM,etc. In some embodiments, memory unit 410 includes more than one type ofmemory. For instance, sensor data may be stored to a hard disk drive andthe computer executable instructions may be stored on a SSD.

In another embodiment, operating environment 400 optionally includesexternal data storage 411 in place of or in addition to data storage410. External data storage 411 is not limited to any particularconfiguration or implementation. Certain embodiments include storage 411being implemented as external hard disk drives, external solid statedrives, and/or data storage capabilities implemented across a computernetwork (e.g. a RAID system accessed across a network). Accordingly,data may be stored and later reviewed independently from the operationof computing device 408. This may also be advantageous in providing databack-ups and possibly storing the sensor information in a more securelocation.

Processing unit 414 illustratively executes instructions stored inmemory unit 410 and controls the operations of operating environment400. Processing unit 414 may include one or more types of devices suchas, but not limited to, application-specific integrated circuits, fieldprogrammable gate arrays, general purpose microprocessors, video/graphiccards, etc. For instance, computing device 408 may include amicroprocessor that processes software or firmware and multiple graphicscards to support multiple display devices 402.

Although embodiments are not limited to any particular configuration ofoperating environment 400, it is worth noting that in one embodimentthat the environment is illustratively made of commercial off-the-shelfcomponents. Or, in other words, the environment can be implementedwithout the need for any specialized hardware. This may be advantageousin that it may allow for at least some embodiments to have lower costs.Some examples, for illustration purposes only and not by limitation, ofcommercial off-the-shelf components that may be used include commercialworkstation class computers capable of running software applications andhosting multiple graphic cards, commercial off-the-shelf graphic cards,standard computer control devices (e.g. keyboards and mice), andcommercial off-the-shelf display monitors such as, but not limited to,flat panel color liquid crystal displays.

FIG. 5 is a process flow diagram of one method of detecting movingobjects. The method shown in FIG. 5 may be used to implement theembodiments described above. Embodiments are not however limited to anyparticular method and illustratively utilize any method of detectingmoving objects. Embodiments illustratively include algorithms or seriesof algorithms that detect and record any change in pixel characteristicsabove a predetermined threshold over time. The detected moving objects(i.e. the portions corresponding to pixels that have changed) are thenhighlighted or otherwise marked such that they are perceptible byoperators.

Returning to FIG. 5, starting at block 502, a reference image isobtained. Additional images of the same area as the reference image arethen obtained at block 504. At block 506, the additional images areregistered or aligned to the reference image. The additional images maybe registered successively one at a time or simultaneously.Additionally, the reference image may be periodically updated by using anew image as the reference image. Also, it is worth noting that theimages need not be a single image from a sensor. In an embodiment, theimages are computed images generated from multiple images of a sensor.For instance, multiple images from one camera may be joined together toform a panoramic view.

At block 508, the additional images are compared to the reference image.Moving objects illustratively correspond to pixels that have changedfrom the reference image. Embodiments are not limited to any particularmethod of change detection. Some examples of testing procedures that canbe utilized to perform change detection operations include linearindependence tests, vectorized tests, and edge motion tests. Further,the change detection can be performed by utilizing application specificinformation such as region of interest or known sizes or shapes. In oneembodiment, a vector change detection algorithm is utilized. In thismanner, a vector method is utilized that determines change at each imagepixel (with respect to a reference image) based on a calculation usingthe test pixel and surrounding pixels in a square region (e.g. 3×3, 5×5,7×7). In another embodiment, the spatial resolution of the changedetection algorithm (the size of the square region) is utilized. Thechange detection operations may be performed simultaneously on multipleframes or performed on one frame at a time on a continuous basis.

At block 510, if one or more changes are detected, the method continuesto block 512. At block 512, several optional events may be triggeredupon a change detection. In one embodiment, the detected moving objectshown on the display is highlighted. For instance, the image shown onthe display is supplemented with a visual indicator (e.g. a box aroundthe moving object). Additionally or alternatively, the portions of analert and tracking window that correspond to the pixels in which changewas detected are highlighted. In yet another embodiment, an audible orvisual indicator is provided to alert operators of detected changes.Such indicators include, for illustration purposes only and not bylimitation, an audio alarm or an indicator light.

Embodiments of the present disclosure may also include dedicatedalgorithms or series of algorithms to classify detected objects. Forinstance, embodiments illustratively identify an unknown object as aperson, a vehicle, an animal, etc., and provide an indication of theclassification to operators.

As has been described above, embodiments of the present disclosureinclude systems and methods for detecting, tracking and displayingmoving objects. Embodiments may provide advantages such as ease of use,increased operator efficiency in detecting objects, and relatively lowercosts to implement. Additionally, certain embodiments may include alertand tracking windows that are intuitively understood by operators andthat provide additional information about detected objects that may nototherwise be available. For instance, alert and tracking windows mayprovide information about relative speed, size, past locations, anddirection of travel. Accordingly, embodiments may be advantageous in awide variety of situations such as, but not limited to, surveillanceand/or security applications.

Finally, it is to be understood that even though numerouscharacteristics and advantages of various embodiments have been setforth in the foregoing description, together with details of thestructure and function of various embodiments, this detailed descriptionis illustrative only, and changes may be made in detail, especially inmatters of structure and arrangements of parts within the principles ofthe present disclosure to the full extent indicated by the broad generalmeaning of the terms in which the appended claims are expressed.

What is claimed is:
 1. A moving object tracking security systemcomprising: a graphical user interface on a display device, thegraphical user interface comprising a plurality of discrete portions,wherein a first portion comprises an alert and tracking window, thealert and tracking window having a first dimension that corresponds to atemporal domain and a second dimension corresponding to a spatialdomain, and a second portion, wherein the second portion provides apictorial display of a plurality of objects, at least one of which is adetected moving object and wherein the detected moving object isindicated by a detection highlight; wherein the graphical user interfaceis configured to simultaneously identify and track each of the pluralityof detected moving objects, wherein simultaneously tracking theplurality of detected moving objects comprises, for each of the detectedmoving objects: providing at least one indicia of a past location of thedetected moving object and a real-time current location of the detectedmoving object; and wherein the current and the past location indicia areprovided simultaneously and wherein the real-time current locationindicia is displayed on both the first and second portion and whereinthe indicia of past location are displayed in the second portion of thegraphical user interface; a processing unit that is a functional part ofthe system that executes computer readable instructions to generate thegraphical user interface and wherein the processing unit generates theindicia of past locations by comparing a location of the detected movingobject in each of a series of successive pictures displayed in thepictorial display portion, wherein each of the successive pictures areregistered successively against a reference image such that the currentlocation of the detected moving object is successively updated andwherein each picture comprises a series of images combined, by theprocessing unit, to form a panoramic view; and wherein the first portioncomprises a plurality of rasters, each raster comprising a row ofpixels, wherein each pixel in the row of pixels corresponds to acorresponding column of pixels in the pictorial image of the secondportion, and wherein each raster is generated by the processing unit asa new image is received and provide on the pictorial display.
 2. Thesystem of claim 1, wherein the detection highlight provides informationabout the real-time current speed, a real-time current location and atleast one past location and at least one past speed for at least one ofa plurality of detected moving objects, and at least one predictedfuture location of the detected moving objects wherein at least one ofthe plurality of detected moving objects does not include any componentof the moving object tracking security system.
 3. The system of claim 1,and further wherein tracking the plurality of detected moving objectsfurther comprises: providing at least one indicia of a predicted futuretrajectory of the detected moving object, and wherein the at least oneindicia of a predicted future trajectory of the detected moving objectis automatically displayed by the graphical user interface and whereinthe processing unit generates the indicia of a predicted futuretrajectory by comparing the registered successive pictures displayed inthe pictorial display portion.
 4. The system of claim 3, whereinreal-time current, past and predicted future locations of the pluralityof detected moving objects within a video window are highlighted withinthe video window utilizing an automated computer algorithm, and whereina notification is provided within the video window, indicating asubstantial change in trajectory or speed from the predicted trajectoryof one of the plurality of detected moving objects.
 5. The system ofclaim 3, wherein the pictorial display portion is a video window, andwherein the video window and the alert and tracking window are alignedin a vertical direction, such that at least one edge of the alert andtracking window is in contact with an edge of the video window.
 6. Thesystem of claim 3, wherein the pictorial display portion is a videowindow, and wherein the video window and the alert and tracking windoware aligned in a horizontal direction, such that at least one edge ofthe alert and tracking window is in contact with an edge of the videowindow.
 7. The system of claim 3, wherein the graphical user interfaceincludes one or more zoom windows in a second portion of the graphicaluser interface, the one or more zoom windows being generated based atleast in part upon a user selection.
 8. The system of claim 3, whereinthe graphical user interface includes one or more zoom windows, the oneor more zoom windows being generated automatically by a softwarealgorithm.
 9. The system of claim 1, wherein the system obtainsinformation from a non-optical sensor.
 10. The moving object trackingsecurity system of claim 1, wherein the alert and tracking window isconfigured to provide an alert when an object passes a specifiedthreshold.
 11. The moving object tracking security system of claim 10,wherein the specified threshold is a distance from an indicatedlocation.
 12. The moving object tracking security system of claim 10,wherein the specified threshold is a minimum altitude.
 13. A method ofdetecting and tracking one or more moving objects, implemented on acomputing device with a processor, the method comprising: displaying acamera image that comprises indicia of one or more objects in a firstgraphical user interface window of a display; detecting, with theprocessor, a potentially moving object within the camera image;determining, with the processor that the potentially moving object is amoving object; tracking, with the processor, the moving object, whereintracking comprises; simultaneously showing, in a second graphical userinterface window of the display, a contrast view of the moving object,wherein the contrast view comprises a plurality of rasters, each rastercomprising a row of pixels corresponding to a compressed view of thedisplayed camera image, wherein the plurality of rasters comprises apath against a background wherein the background comprises a first colorand the path comprises a second color, wherein the path comprises aplurality of successively plotted past locations and ending with areal-time current location of the moving object; generating theplurality of rasters, wherein generating a raster comprises using theprocessor to successively compare a first position of the moving objectin a first camera image with a second position of the moving object in asecond camera image, and highlighting a pixel corresponding to a portionof the compressed view containing a detected moving object; updating thesecond graphical user interface after each successive comparison byadding a new raster representative of a newly detected real-time currentposition of the moving object in a last-taken camera image, and whereina new raster is added to the plurality of rasters such that, over time,the path grows as each new raster is added, wherein the new rasterrepresentative of the newly detected real-time current position of themoving object is determined by an algorithm, wherein the algorithm isconfigured to analyze each pixel in the raster, detects and records achange in pixel characteristics above a threshold; and wherein the firstand second graphical user interface windows are provided in separateviewing windows such that they may be viewed simultaneously.
 14. Themethod of claim 13, wherein the first and the second graphical userinterfaces are shown on one display device.
 15. The method of claim 13,wherein the first and second graphical user interfaces are shown onmultiple display devices.
 16. The method of claim 13, and furthercomprising: providing a larger representation of the moving object in azoom window.
 17. The method of claim 16, further comprising: changing amagnification level of the zoom window.
 18. The method of claim 16,wherein the zoom window and the first graphical user interface windoware coded such that an area of the first graphical user interface windowthat corresponds to the zoom window is identifiable.
 19. The method ofclaim 13, wherein the moving object is a first moving object and thepath is a first path and wherein the method further comprises:simultaneously showing, in the second graphical user interface window, acontrast view of the first moving object as well as a second movingobject, wherein the contrast view comprises the first path as well as asecond path against the background, wherein the first path comprises aplurality of successively plotted past locations and ending with areal-time current location of the first moving object, and wherein thesecond path comprises a plurality of successively plotted past locationsand ending with a real-time current location of the second moving objectand wherein the plurality of past locations of the first and secondmoving objects are generated using the processor to compare a firstposition of each of the moving objects in the first camera image with asecond position of the moving objects in a second later-taken cameraimage.
 20. A moving object tracking system, implemented on a computerwith a processing unit comprising: a display comprising: a first windowthat includes a camera image of an area presented on the display, thecamera image of the area having within the camera image a plurality ofdetected moving objects, wherein the camera image is periodicallyupdated such that the first window displays a real-time current positionof the plurality of detected moving objects; a second window thatincludes information about a past location, a real-time currentlocation, and an estimated future location for each of the plurality ofmoving detected objects, wherein the information about the pastlocations comprise a path presented against a contrasting background,and wherein the real-time current location comprises an indicator at anend of the path; a third window that provides a zoom view of one of theplurality of detected moving objects along with an indication of theportion of the first or second window being shown in the third window;and wherein the first and second windows are presented simultaneouslyand wherein the second window updates automatically based on acomparison of a series of changing positions of each of the plurality ofdetected moving objects in the camera image; and wherein the processingunit is a functional part of the system that executes computer readableinstructions to generate the first, second and third windows on thedisplay and wherein the processing unit is configured to receive aseries of successive camera images, analyze the series of successivecamera images by identifying a change in position of a moving objectbetween two successive camera images, and generate a raster indicativeof the detected motion, wherein the processing unit is configured todisplay the camera image in the first window of the display, and furtherconfigured to update the second window by adding the generated raster toa series of previously generated rasters.
 21. The system of claim 20,wherein the second window has one dimension that corresponds to aspatial dimension of the first window.
 22. The system of claim 20,further comprising: a data storage unit that stores information aboutthe area collected by a sensor; and playback controls that allow a userto review the stored information.
 23. The system of claim 20, furthercomprising: an interface that receives information about the area froman external data store; and playback controls that allow a user toreview the received information.
 24. The system of claim 20, wherein thefirst window and the second window are updated at the same rate.
 25. Thesystem of claim 20, wherein the first window and the second window areupdated at different rates.
 26. The moving object tracking system ofclaim 20, wherein the camera image is a panoramic view comprisingmultiple images joined together.